Animated and computer-generated cartoons have become quite sophisticated. Some full-length animated motion pictures starring animated characters have generated millions of dollars in revenue from ticket sales and sales of licensed merchandise. The characters in these cartoons and movies usually move and talk realistically. At least part of the success of these movies can be attributed to this life-like motion of the characters.
Synchronizing the mouth features of a speaking animated character to the speech of the character is particularly difficult. Poor synchronization can result in characters appearing as though they were in a poorly dubbed foreign film. Proper synchronization of the mouth features of a character to the speech of the character can be difficult and expensive to achieve.
The mouth features of an animated character can be described by two attributes: the position of the lips, i.e., lip position, and the amount of opening between the lips, i.e., mouth opening. Sometimes, an animator draws the mouth features of an animated character by examining his face in a mirror to determine his lip position and the mouth opening as he speaks the words that the character is to speak. This process of drawing the lip position and mouth opening of an animated character can be time-consuming. In addition, this process can result in an inaccurate representation of speech.
For instance, if the animated sequence contains 10 frames or cells per second, then the animator must estimate the character's lip position and mouth opening at one-tenth of a second intervals to achieve synchronization. This estimation requires a great deal of experience to perfect and, even with experience, this process can result in poor synchronization. In addition, this process can be time-consuming and expensive if the animator must redraw the mouth features to synchronize them with the character's speech. Thus, there is the need in the art for a method for determining the lip position and the opening between the lips of a speaking animated character that is quick, efficient and accurate.
Speaking characters are not only seen in cartoons and motion pictures. For example, talking mechanical, or stuffed, characters are popular, especially with children. The problems of synchronizing the lip position and mouth opening of a talking mechanical character are in many ways similar to the problems of synchronizing the lip position and mouth opening of a cartoon character. For instance, poor synchronization may result in the mechanical character's mouth appearing to open and close like a mousetrap rather than like a mouth of a human being. Thus, there is the need in the art for a quick, efficient and effective method for determining the lip position and the opening between the lips of a speaking mechanical character.
One method that has been used to determine the mouth opening of a speaking mechanical character is integrating over time the time-domain voice signal that the mechanical character is to speak. The result of this integration is stored in a capacitor and used as a rough approximation of the amount of opening between the lips of the mechanical character. One disadvantage of this method is that it only gives a rough approximation of how wide the mouth of the character should be opened, resulting in a coarse granularity that may appear as a simple opening and closing of the mouth of the mechanical character. Another disadvantage of this method is that this method does not provide any information about the position of the lips of the mechanical character. For example, the lips determine whether someone is pronouncing an "a" or a "t" sound. Without defining lip position, the synchronization of the mouth features to the speech of the mechanical character is not fully realized. Still another disadvantage is that this method requires discrete analog components, such as capacitors, that are not easily compatible with a digital environment.
Therefore, there is a need in the art for a quick, efficient and accurate method for determining lip position and mouth opening for both mechanical and animated characters. There is a further need for a method for determining lip position and mouth opening that has a fine granularity, i.e., provides an accurate representation of lip position and mouth opening. There is a further need for a method for determining lip position and mouth opening that is compatible with a digital environment. There is still a further need for a method for synchronizing the mouth features of an animated or mechanical character to the speech of the character that takes into account not only the amount of opening between the lips of the character, but also the position of the lips.